Shore Side Data System (600031 and 900823) : Data Services

searchDataStream

Background

A service was need to search data streams for some specific text. This service does not apply to all instruments as not all instruments output ASCII in their packets, but it was deemed handy to have for those that do output ASCII.

Parameter	Description	Options	Default Value	Required
Device ID	The SSDS ID of the device whose data stream the user wants to search	Any SSDS ID	N/A	Y
Expression	This is the regular expression (Java style) that will be used to search for	Any valid regular expression	N/A	Y
PacketOrLine	This is an option that specifies if the service should return the whole packet contents if a match is found, or if just the line out of the packet is found	PACKET or LINE	PACKET	N

The Interface

searchDataStream

// Packet or line enumerations
public final static String RETURN_PACKET = "packet";
public final static String RETURN_LINE = "line";

searchDataStream(
     Long deviceID,                     // The ID of the Device whose data stream will be searched
     String expression,                 // The (Java style) regular expression to look for in the
                                        // data stream
     packetOrLine                       // The flag to indicate if the user wants the whole packet
                                        // from the match returned, or just the line
)

The call will return a collection of SSDSDevicePackets

Java EJB client

Under construction

REST client

If you want to use the REST-style interface the HTTP call would looks something like:

http://new-ssds.mbari.org:8080/servlet/DataAccessServlet?objectToInvokeOn=SQLDataStreamRawDataAccess&method=getDataStreamProperties
&p1Type=Long&p1Value=1300
&p2Type=Long&p2Value=1
&p3Type=Boolean&p3Value=true
&p4Type=Date&p4Value=2008-12-00T00:00:00Z
&p5Type=Date&p5Value=2008-01-00T00:00:00Z
&p6Type=String&p6Value=timeGap
&p7Type=Long&p7Value=5000
&p8Type=String&p8Value=userSpecified
&p9Type=Long&p9Value=10
&p10Type=Date&p10Value=
&p11Type=Date&p11Value=
&p12Type=Long&p12Value=10000

And the return might look something like

java.util.Properties{}

Web Service Client

Under construction

getDataStreamProperties

Background

The desire it to have a service that can characterize a DataStream from an instrument. This would allow for easier monitoring of instruments on the network.

Parameter	Description	Options	Default Value	Required
Device ID	The SSDS ID of the device that the user wants information about	Any SSDS ID	N/A	Y
Number Of Samples To Average	The number of samples back that the service is to use to calculate the average sampling interval	Any Number	10	N
Check for gaps	Is a flag that tells the service to try to find data gaps based on some average sample interval	True/False	False	N
Gap Interval	Is the number of milliseconds to use as the sampling interval to search for gaps. If this is specified it will override the average gap calculated by the service	Any number of milliseconds	N/A	N

The Interface

The things that the user would want to know are:

Return	Parameters
Date and time of last packet received	Device ID RecordType to search for (nothing/default means most recent packet)
Total Number Of Records	Device ID RecordType to search for (nothing/default specified means all packets)
Data Gaps	Device ID RecordType Time window over data to search for gaps Gap criteria Type of gap Time only Sequence number only Time and sequence number Ways to specify gap Margin on gap in milliseconds (anything longer than gap + margin will be considered a possible gap) Let service calculate gap constraints (calculate average time between sample) Number of points to use (points back from most recent packet) Or time window to use (start to end time) Specify gap constraints Gap in milliseconds

So the API interface looks like:

getDataStreamProperties

// Type of criteria to use for finding gaps
public final static String TIME_ONLY_GAP = "timeGap";
public final static String SEQ_ONLY_GAP = "seqGap";
public final static String TIME_SEQ_GAP = "timeSeqGap";

// The method of specifying the gap
public final static String SERVICE_CALCULATED = "serviceCalculated";
public final static String USER_SPECIFIED = "userSpecified";

getDataStreamProperties(
     Long deviceID,                     // The ID of the Device to get the properties for
     Long recordType,                   // The RecordType that will be singled out (devices
                                        // can send out more than one RecordType)
                                        // 0 = Metadata Packets
                                        // 1+ = Device specific record types
     Boolean checkForGaps,              // A Boolean that indicates if the caller wants to
                                        // have the service check for data gaps (true means
                                        // the service will check for gaps and false/null
                                        // means it will not
     Date startGapCheckWindow,          // The start date of the window over which to search for
                                        // gaps.
     Date endGapCheckWindow,            // The end date of the window over which to search for gaps.
     String typeOfGap,                  // One of three types: "timeGap", "seqGap", "timeSeqGap"
     Long marginMillis,                 // This is the number of milliseconds that are used as 'slop'
                                        // around the specification for a gap. In other words, if
                                        // this is > 0, the service will consider any time between 
                                        // samples that is less than the specified gap plus this margin,
                                        // it will assume that it is not a gap condition.  This is to
                                        // prevent false positives when the sample timestamps aren't exactly
                                        // on the interval.
     String gapSpec,                    // There are two ways to specify a gap: 
                                        // "serviceCalculated" or "userSpecified"
     Long numberOfRecords,              // If the call specifies SERVICE_CALCULATED and this
                                        // is greater than 0, the service will use 'numberOfRecords'
                                        // most recent records of the specified RecordType
                                        // in calculating the average time between samples
     Date intervalCalcStartWindow,      // This is the date that starts the window over which the data
                                        // will be used to calculate the average time between samples
                                        // NOTE: If numberOfRecords is specified, this is ignored.
     Date intervalCalcEndWindow,        // This is the date that ends the window over which the data
                                        // will be used to calculate the average time between samples
                                        // NOTE: If numberOfRecords is specified, this will be used as
                                        // the endtime and then the service will use the numberOfRecords
                                        // before this time as the data to calculate the average
                                        // time interval.
     Long gapInMillis                   // If the gapSpec is USER_SPECIFIED, then the service will use
                                        // this number of milliseconds as the gap for identifying gaps.
)

With a return that has the format of:
Properties Objects with properties:

Property Name	Value
lastPacketDateTime	This is the date and time of the last packet received
totalNumberOfRecords	This is the total number of records for the parameters specified
numberOfFuturePackets	This is the number of packet that appear in the future. This should be zero and if they are not, there could be bad data
averagSampleIntervalInMillis	This is the number of milliseconds that the service used to find data gaps
marginInMillis	This is the number of milliseconds as a margin that the service used to find data gaps
numRecordsSearchedForGaps	This is the number of records that were searched through while trying to find gaps using the gap criteria
dataGap1Start	This is the date and time of start of the first possible gap in the data
dataGap1End	This is the date and time of end of the first possible gap in the data
.	.
.	.
.	.
dataGapNStart	This is the date and time of start of the Nth possible gap in the data
dataGapNEnd	This is the date and time of end of the Nth possible gap in the data

Java EJB client

Under construction

REST client

If you want to use the REST-style interface the HTTP call would looks something like:

http://new-ssds.mbari.org:8080/servlet/DataAccessServlet?objectToInvokeOn=SQLDataStreamRawDataAccess&method=getDataStreamProperties
&p1Type=Long&p1Value=1300
&p2Type=Long&p2Value=1
&p3Type=Boolean&p3Value=true
&p4Type=Date&p4Value=2008-12-00T00:00:00Z
&p5Type=Date&p5Value=2008-01-00T00:00:00Z
&p6Type=String&p6Value=timeGap
&p7Type=Long&p7Value=5000
&p8Type=String&p8Value=userSpecified
&p9Type=Long&p9Value=10
&p10Type=Date&p10Value=
&p11Type=Date&p11Value=
&p12Type=Long&p12Value=10000

And the return might look something like

java.util.Properties{}

Web Service Client

Under construction